Papers with neural network model
Neural Fine-Grained Entity Type Classification with Hierarchy-Aware Loss (N18-1)
Copied to clipboard
| Challenge: | Existing methods for fine-grained type classification rely on distant supervision and are susceptible to noisy labels that can be out-of-context or overly-specific. |
| Approach: | They propose a neural network model that uses cross-entropy loss function to handle out-of-context labels and hierarchical loss normalization to cope with overly-specific ones. |
| Outcome: | The proposed model outperforms the state-of-the-art on established benchmarks for the task. |
Fine-grained Interpretation and Causation Analysis in Deep NLP Models (2021.naacl-tutorials)
Copied to clipboard
| Challenge: | Despite the proven efficacy of deep neural networks at-large, their opaqueness is a major cause of concern. |
| Approach: | They will present research work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
| Outcome: | This paper presents work on interpreting fine-grained components of a neural network model from two perspectives, i) fine-grain interpretation, and ii) causation analysis. |
DeClarE: Debunking Fake News and False Claims using Evidence-Aware Deep Learning (D18-1)
Copied to clipboard
| Challenge: | Recent work on automated fact-checking does not consider external evidence, but requires rich lexicons. |
| Approach: | They propose a neural network model that aggregates external evidence and language . they also derive informative features for generating user-comprehensible explanations . |
| Outcome: | The proposed model aggregates signals from external evidence articles, language and trustworthiness of their sources without human intervention. |
Neural language models as psycholinguistic subjects: Representations of syntactic state (N19-1)
Copied to clipboard
| Challenge: | a recent study examines the extent to which neural network language models reflect incremental representations of syntactic state . we examine neural network model behavior on sentences chosen to probe specific aspects of the learned representations . |
| Approach: | They employ experimental methodologies developed in psycholinguistics to study syntactic representation in the human mind. |
| Outcome: | The proposed models are trained on large datasets and only sensitive to subtle cues . the results raise questions about the accuracy of the models and their performance . |
Rapid Customization for Event Extraction (P19-3)
Copied to clipboard
| Challenge: | a novel system allows users to customize event extraction to find new event types and their arguments. |
| Approach: | They propose a system that allows a user to find, expand and filter event triggers by exploring an unannotated development corpus. |
| Outcome: | The proposed system can find, expand and filter event triggers from an unannotated development corpus . it trains a generic argument attachment model for extracting Actor, Place, and Time . |
Representation Learning of Entities and Documents from Knowledge Base Descriptions (C18-1)
Copied to clipboard
| Challenge: | Using documents extracted from Wikipedia, we train a neural network model that learns distributed representations of entities and documents directly from a knowledge base. |
| Approach: | They propose a neural network model that learns distributed representations of entities from a knowledge base. |
| Outcome: | The proposed model performs state-of-the-art on fine-grained entity typing and multiclass text classification tasks. |
Bacteria Biotope Relation Extraction via Lexical Chains and Dependency Graphs (D19-57)
Copied to clipboard
| Challenge: | In this paper, we describe our approach for the Bacteria Biotopes relation extraction subtask in the BioNLP Shared Task 2019 . |
| Approach: | They propose a novel approach for dependency graph construction based on lexical chains . they then propose 'neuro network' model which uses short-term memories and syntax information . |
| Outcome: | The proposed approach achieves the best F1 (66.3%) in the official evaluation participated by 7 teams. |
Embedding WordNet Knowledge for Textual Entailment (C18-1)
Copied to clipboard
| Challenge: | Existing deep learning models for textual entailment do not require any feature engineering or linguistic analysis. |
| Approach: | They propose to embed WordNet-derived lexical entailment relations into specially-learned word vectors and incorporate them into a decomposable attention model for textual enlightment. |
| Outcome: | The proposed model significantly improves on the SICK and SNLI datasets. |
Demand-Weighted Completeness Prediction for a Knowledge Base (N18-3)
Copied to clipboard
| Challenge: | Knowledge Bases (KBs) are widely used for representing information in a structured format. |
| Approach: | They propose a method to measure Demand-Weighted Completeness by defining an entity by its classes and using usage data to predict relation distributions. |
| Outcome: | The proposed method can be used to estimate completeness of knowledge bases based on how they are used and can quantify usage and completeness changes over time. |
A Knowledge-Augmented Neural Network Model for Implicit Discourse Relation Classification (C18-1)
Copied to clipboard
| Challenge: | Existing studies on implicit discourse relation classification have shown success using feedforward networks and convolutional neural networks. |
| Approach: | They propose to augment input text with external knowledge and context and adopt a neural network model that can effectively handle the augmented text. |
| Outcome: | The proposed model outperforms existing models on implicit discourse relation classification. |
From Spelling to Grammar: A New Framework for Chinese Grammatical Error Correction (2022.findings-emnlp)
Copied to clipboard
| Challenge: | Chinese Grammatical Error Correction (CGEC) aims to generate correct sentences from erroneous sequences. |
| Approach: | They propose a zero-shot approach for spelling error correction that is simple but effective . they propose auxiliary task to predict POS sequence of target sentence . |
| Outcome: | The proposed framework achieves 42.11 F-0.5 on the English GEC dataset outperforms the previous state-of-the-art by a wide margin of 1.30 points. |
Integrated Directional Gradients: Feature Interaction Attribution for Neural NLP Models (2021.acl-long)
Copied to clipboard
| Challenge: | Existing methods for attribution of importance to features borrowed from cooperative game theory . success of Deep Neural Networks has led to their ability to learn from complex higher order interactions from raw features. |
| Approach: | They propose a method for attributing importance scores to groups of features . they propose axioms that any intuitive feature group attribution method should satisfy . |
| Outcome: | The proposed method captures the importance of features in a linguistic model using negations and conjunctions. |
How do we get there? Evaluating transformer neural networks as cognitive models for English past tense inflection (2022.aacl-main)
Copied to clipboard
| Challenge: | English past tense inflections is a typical quasi-regularity task, but it is criticized that it learns only to generalize the most frequent pattern, but not the regular pattern. |
| Approach: | They train a set of transformer models with different settings to examine their behavior on a typical English quasi-regularity task. |
| Outcome: | The models achieved high accuracy on unseen regular verbs and some accuracy on unseen irregular verbs. |
Understanding User Utterances in a Dialog System for Caregiving (2020.lrec-1)
Copied to clipboard
| Challenge: | a dialog system that can monitor the health status of seniors has a huge potential for solving the labor shortage in the caregiving industry in aging societies. |
| Approach: | They are developing a yes/no response classifier and an entailment recognizer to correctly interpret user utterances. |
| Outcome: | The proposed system can correctly interpret user utterances and can monitor the health of seniors. |
Numerical Optimizations for Weighted Low-rank Estimation on Language Models (2022.emnlp-main)
Copied to clipboard
| Challenge: | Singular value decomposition (SVD) is one of the most popular methods for estimating a target matrix with smaller matrices. |
| Approach: | They propose a method that approximates a target matrix with smaller matrices by two smaller . they also propose metric to predict when the SVD may introduce a significant performance drop. |
| Outcome: | The proposed method can perform better than current SOTA methods in compressing Transformer-based language models. |
Text Categorization by Learning Predominant Sense of Words as Auxiliary Task (P19-1)
Copied to clipboard
| Challenge: | Existing methods for text categorization use implicit representations to learn the senses of words. |
| Approach: | They propose a method for text categorization by leveraging the predominant sense of words depending on the domain. |
| Outcome: | The proposed model improves performance on four benchmark datasets. |
Rumor Detection by Exploiting User Credibility Information, Attention and Multi-task Learning (P19-1)
Copied to clipboard
| Challenge: | Social media platforms do not always pose authentic information, and rumors spread fear or hate. |
| Approach: | They propose a new multi-task learning approach for rumor detection and stance classification tasks. |
| Outcome: | The proposed model outperforms the state-of-the-art rumor detection approaches on two datasets. |
Chinese Discourse Parsing: Model and Evaluation (2020.lrec-1)
Copied to clipboard
| Challenge: | Chinese discourse parsing has not yet a consistent evaluation metric . micro vs. macro F1 scores, binary v. multiway ground truth, and left-heavy v . right-heaviness binarization are important for Chinese discourses . |
| Approach: | They propose a neural network model that unifies a pre-trained transformer and a CKY-like algorithm and compare it with previous models with different evaluation scenarios. |
| Outcome: | The proposed model outperforms the previous models with different evaluation scenarios. |
Multi-Channel Convolutional Neural Network for Twitter Emotion and Sentiment Recognition (N19-1)
Copied to clipboard
| Challenge: | Existing methods to analyze tweets are based on lexical features and a multi-channel convolutional neural architecture. |
| Approach: | They propose a neural network which can use different emotion and sentiment indicators such as hashtags, emoticons and emojis present in tweets to improve the performance of emotion and feelings identification. |
| Outcome: | The proposed model can use hashtags, emoticons and emojis present in tweets and improves emotion and sentiment identification. |
Learning to Rank Question-Answer Pairs Using Hierarchical Recurrent Encoder with Latent Topic Clustering (N18-1)
Copied to clipboard
| Challenge: | Existing models for sentence pair ranking are based on hierarchical recurrent neural network and latent topic clustering module. |
| Approach: | They propose a hierarchical recurrent neural network and latent topic clustering module to adapt a recursive hierarchic neural network to rank candidate answers. |
| Outcome: | The proposed model shows small performance degradations in longer text comprehension compared to current models which suffer from it. |
Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues (N19-1)
Copied to clipboard
| Challenge: | Recent surge of text-based online counseling applications enables us to collect and analyze interactions between counselors and clients. |
| Approach: | They develop a pre-trained conversation model that learns to classify client utterances into categories that help counselors in diagnosing client status and predicting counseling outcome. |
| Outcome: | The proposed model outperforms state-of-the-art comparison models and shows expected linguistic patterns for each category. |
Spotting Spurious Data with Neural Networks (N18-1)
Copied to clipboard
| Challenge: | Existing methods to identify spurious instances require either annotations generated by each individual annotator or both task-specific and instance-type annotations. |
| Approach: | They propose an approach that discriminates instances based on their "difficulty to learn" they use queueing theory and psychology of learning to improve annotations . |
| Outcome: | The proposed methods outperform state-of-the-art baselines and have a MAP of 0.85 and 0.22 in identifying spurious instances in synthetic and carefully-crowdsourced real-world datasets respectively. |
Calibrating Structured Output Predictors for Natural Language Processing (2020.acl-main)
Copied to clipboard
| Challenge: | Several modern machine-learning based NLP systems can provide a confidence score with their output predictions. |
| Approach: | They propose a general calibration scheme for output entities of interest in NLP applications that can be used to calibrate confidence scores. |
| Outcome: | The proposed calibration scheme outperforms current calibration techniques for Named Entity Recognition, Part-of-speech tagging and Question Answering systems. |
Correcting Chinese Word Usage Errors for Learning Chinese as a Second Language (C18-1)
Copied to clipboard
| Challenge: | a word usage error is the most common error type in Chinese, according to the HSK dynamic composition corpus . a system that considers both target erroneous token and context can generate a correction vector . |
| Approach: | They propose a neural network model that considers target erroneous token and context to generate a correction vector and compare it against a candidate vocabulary to propose suitable corrections. |
| Outcome: | The proposed model can detect 91% of the cases and propose suitable corrections within a list of five candidates. |
Folksonomication: Predicting Tags for Movies from Plot Synopses using Emotion Flow Encoded Neural Network (C18-1)
Copied to clipboard
| Challenge: | Existing systems that generate tags for movies can help users better retrieve movies based on their personal preferences and user profiles. |
| Approach: | They propose a neural network model that merges synopses and emotion flows to predict a set of movies' tags. |
| Outcome: | The proposed model outperforms a machine learning system by learning 18% more tags than the previous one. |
Predicting Human Activities from User-Generated Content (P19-1)
Copied to clipboard
| Challenge: | Several studies have applied computational approaches to the understanding and modeling of human behavior at scale and in real time. |
| Approach: | They propose a sentence embedding framework tailored to recognize the semantics of human activities and perform automatic clustering of these activities. |
| Outcome: | The proposed framework can make predictions based on the text of user-generated content and self-description. |
Learning to Ask Good Questions: Ranking Clarification Questions using Neural Expected Value of Perfect Information (P18-1)
Copied to clipboard
| Challenge: | StackExchange users routinely ask clarifying questions to fill information gaps . a principle goal of asking questions is to fill this information gap . |
| Approach: | They build a model to rank candidates by their usefulness to a given post . they use data from StackExchange to evaluate the model against human judgments . |
| Outcome: | The proposed model outperforms baselines on 500 samples of StackExchange's clarification questions. |
Predicting Degrees of Technicality in Automatic Terminology Extraction (2020.acl-main)
Copied to clipboard
| Challenge: | a recent study has focused on term technicality, but there are still few studies on it. |
| Approach: | They semi-automatically create a German gold standard of technicality across four domains . they propose two new models to exploit general- vs. domain-specific comparisons based on vector spaces . |
| Outcome: | The proposed model outperforms previous methods in terms of general- vs. domain-specific comparisons. |
Dual Attention Model for Citation Recommendation (2020.coling-main)
Copied to clipboard
| Challenge: | Existing methods for recommending citations suffer from severe information loss . citation recommender methods do not consider the section of the paper for which the user is writing and for which they need to find a citation . |
| Approach: | They propose a novel embedding-based neural network to recommend citations during manuscript preparation. |
| Outcome: | The proposed method can recommend citations during manuscript preparation. |
Document-Level Event Factuality Identification via Adversarial Neural Network (N19-1)
Copied to clipboard
| Challenge: | Document-level event factuality identification is crucial for discourse understanding in NLP . identifying document-level factual of events requires comprehensive understanding of documents . |
| Approach: | They propose to construct a corpus annotated with document- and sentence-level event factuality information on English and Chinese texts. |
| Outcome: | The proposed model outperforms baselines on the constructed corpus. |
A Myanmar (Burmese)-English Named Entity Transliteration Dictionary (2020.lrec-1)
Copied to clipboard
| Challenge: | Currently, there are no data available for the transcription of borrowed English words in Myanmar . lack of resources is a problem for many understudied languages . |
| Approach: | They construct a dictionary of Myanmar-English transliteration instances using a CC BY-NC-SA license. |
| Outcome: | The proposed model outperforms the statistical model significantly on the character level. |
Block-wise Word Embedding Compression Revisited: Better Weighting and Structuring (2021.findings-emnlp)
Copied to clipboard
| Challenge: | Existing methods for word embedding compression are limited . word embeds have a considerable size and need to be compressed to deploy on edge devices . |
| Approach: | They propose a block-wise low-rank approximation method for word embedding called GroupReduce . they propose 'frequency-inverse document frequency method' and a differentiable method for weighting . |
| Outcome: | The proposed algorithm more effectively finds word weights than competitors in most cases. |
What Would a Teacher Do? Predicting Future Talk Moves (2021.findings-acl)
Copied to clipboard
| Challenge: | Recent advances in natural language processing (NLP) have the ability to transform how classroom learning takes place. |
| Approach: | They propose a task that uses the academically productive talk framework to learn strategies that make for the best learning experience. |
| Outcome: | The proposed task outperforms baselines on academically productive talk (FTMP) and shows that it outperformed human performance on FTMP. |
Fine-grained Entity Typing without Knowledge Base (2021.emnlp-main)
Copied to clipboard
| Challenge: | Existing work on fine-grained entity typing (FET) relies on knowledge bases as distant supervision, but lack of or incompleteness of KB can hinder training. |
| Approach: | They propose a two-step framework that trains FET models without accessing any knowledge base. |
| Outcome: | The proposed framework achieves competitive performance with respect to the models trained on the original KB-supervised datasets. |
A Neural Network Model for Part-Of-Speech Tagging of Social Media Texts (L18-1)
Copied to clipboard
| Challenge: | Recent approaches based on end-to-end Deep Neural Networks (DNNs) have shown promising results for Natural Language Processing (NLP). |
| Approach: | They propose a neural network model for part-of-speech (POS) tagging of User-Generated Content (UGC) such as Twitter, Facebook and Web forums that uses character and word representations. |
| Outcome: | The proposed model is end-to-end and uses character and word representations . it is compared with existing models on social media in English, german, french, italian and spanish . |
NeuInfer: Knowledge Inference on N-ary Facts (2020.acl-main)
Copied to clipboard
| Challenge: | Existing studies on knowledge inference on binary facts have focused on finding out connotative valid facts. |
| Approach: | They propose a neural network model, NeuInfer, for knowledge inference on n-ary facts. |
| Outcome: | The proposed model can cope with the task to infer an unknown element in a whole fact, while ignoring the binary facts. |
Linguistically-Informed Self-Attention for Semantic Role Labeling (D18-1)
Copied to clipboard
| Challenge: | Existing models of semantic role labeling use no explicit linguistic features. prior work has shown that syntax trees can dramatically improve SRL decoding. |
| Approach: | They propose a neural network model that incorporates syntax using only raw tokens . they show that LISA out-performs the state-of-the-art with contextually-encoded word representations a 1.0 F1 on newswire and 2.0 F1 in out-of domain text . |
| Outcome: | The proposed model outperforms the state-of-the-art model with word embeddings and predicted predicates. |
MCˆ2: Multi-perspective Convolutional Cube for Conversational Machine Reading Comprehension (P19-1)
Copied to clipboard
| Challenge: | Existing models combine previous questions for conversation understanding and only employ recurrent neural networks (RNN) for reasoning. |
| Approach: | They propose a multi-perspective convolutional cube model that integrates 1D and 2D convolutions with recurrent neural networks (RNN) to understand context from different perspectives. |
| Outcome: | The proposed model is based on the Conversational Question Answering (CoQA) dataset and achieves state-of-the-art results. |
STANCY: Stance Classification Based on Consistency Cues (D19-1)
Copied to clipboard
| Challenge: | Recent work has shown that stance classification is a critical step for information credibility and automated fact-checking. |
| Approach: | They propose a neural network model for stance classification leveraging BERT representations and augmenting them with a novel consistency constraint. |
| Outcome: | The proposed model outperforms existing methods on a Perspectrum dataset and shows that it is more accurate than existing methods. |